feat: Software Factory Harness - observability, reliability and gate loop - #7
Open
mmunisamy-emergence wants to merge 1 commit into
Open
feat: Software Factory Harness - observability, reliability and gate loop#7mmunisamy-emergence wants to merge 1 commit into
mmunisamy-emergence wants to merge 1 commit into
Conversation
Renames `orchestrator/` → `harness/` and builds the complete Dark Factory control plane on top of the existing provenance foundation. ## Harness (deterministic step loop) - `harness/harness.py` — owns research→plan→implement→gate-loop→PR cycle; calls each skill individually via subprocess, reads structured gate JSON, retries with /fix-failures; integrates all reliability primitives below ## Reliability primitives - `harness/scrubber.py` — regex scrubs secrets from provenance output - `harness/locks.py` — fcntl per-repo file lock; prevents concurrent runs - `harness/checkpoint.py` — atomic checkpoint after each step; crash resume - `harness/circuit_breaker.py` — opens after 5 cross-run gate failures; CLI reset - `harness/watchdog.py` — background thread; warns at 15–45 min, kills at limit ## Observability server - `harness/server.py` — FastAPI; 14 REST+SSE endpoints, hand-rolled Prometheus /metrics (no prometheus_client dep); auto-starts via PID file ## User interfaces - `harness/tui.py` — Rich terminal dashboard (operators); live refresh, gate health bars, key bindings (watch/cancel/refresh) - `harness/dashboard/index.html` — vanilla-JS web dashboard (team leads); SSE live updates, gate heatmap, run detail panel, circuit-breaker banner ## Provenance enhancements - `harness/provenance.py` — adds cost_usd/tokens_in/tokens_out per event, flakiness detection (pass without fix), rotate() archival, total_cost(), flaky_gates() analysis helpers ## Verification gate skills - `skills/run-linter/SKILL.md` — auto-fix pass then errors; writes JSON result - `skills/run-tests/SKILL.md` — full suite excl. evals/; writes JSON result - `skills/run-evals/SKILL.md` — acceptance-criteria tests; writes JSON result - `skills/run-code-review/SKILL.md` — blockers-only verdict; writes JSON result - `skills/fix-failures/SKILL.md` — reads failures JSON, applies targeted fixes - `skills/verify-and-fix/SKILL.md` — standalone 4-gate retry loop (skill mode) ## CLI additions (python -m harness <cmd>) watch, queue, cancel, resume, circuit-breaker status/reset, tui, server, cost, runs, provenance stats/runs/export
mmunisamy-emergence
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
orchestrator/toharness/(Dark Factory execution engine)Changes
Harness (deterministic step loop)
harness/harness.pyowns research -> plan -> implement -> gate-loop -> PR; calls each skill individually, reads structured gate JSON, retries with /fix-failures; integrates all reliability primitives.Reliability primitives
harness/scrubber.py— regex scrubs secrets from provenance output before writeharness/locks.py— fcntl per-repo file lock; prevents concurrent runs on same repoharness/checkpoint.py— atomic checkpoint after each step; survives crashes; resume supportharness/circuit_breaker.py— opens after 5 cross-run gate failures; CLI resetharness/watchdog.py— background thread; warns at 15-45 min, kills at hard limitObservability server
harness/server.py— FastAPI; 14 REST + SSE endpoints; hand-rolled Prometheus /metrics (no prometheus_client dep); auto-starts via PID file on first harness implement run.User interfaces
harness/tui.py— Rich terminal dashboard for operators; live refresh, gate health bars, key bindings (w=watch, c=cancel, r=refresh)harness/dashboard/index.html— vanilla-JS web dashboard for team leads; SSE live updates, 7-day gate health heatmap, run detail panel, circuit-breaker alert bannerProvenance enhancements
harness/provenance.pyadds cost_usd/tokens_in/tokens_out per event; flakiness detection; rotate() archival; total_cost(), flaky_gates() analysis helpers.Verification gate skills
skills/run-linter/— auto-fix pass then errors; writes structured JSON resultskills/run-tests/— full suite excl. tests/evals/; writes JSON resultskills/run-evals/— acceptance-criteria tests per issue; writes JSON resultskills/run-code-review/— blockers-only parallel review; writes JSON resultskills/fix-failures/— reads failures JSON, applies targeted fixes per gate typeskills/verify-and-fix/— standalone 4-gate retry loop for skill-mode useNew CLI commands
watch, queue, cancel, resume, circuit-breaker status/reset, tui, server, cost, runs
Test plan
gh pr create
--title "feat: Dark Factory harness — observability, reliability & gate loop"
--body-file /tmp/pr_body.md